Surface abnormality detection device and system

ABSTRACT

There is provided a surface abnormality detection device, and a system, capable of detecting an abnormal portion having a displacement below the distance measurement accuracy when detecting the abnormal portion on the surface of a structure. A surface abnormality detection device includes a classification means for classifying an object under measurement into one or more clusters having the same structure, based on position information at a plurality of points on a surface of the object under measurement; a determination means for determining a reflection brightness normal value of the cluster based on a distribution of reflection brightness values at a plurality of points on a surface of the cluster; and an identification means for identifying an abnormal portion on the surface of the cluster based on a difference between the reflection brightness normal value and the reflection brightness value at each of the plurality of points.

TECHNICAL FIELD

The present disclosure relates to a surface abnormality detection deviceand a system, and more particularly, to a surface abnormality detectiondevice, and a system, capable of detecting an abnormal portion having adisplacement below the distance measurement accuracy when detecting theabnormal portion on the surface of a structure.

BACKGROUND ART

In complex equipment in a facility, a portion of deterioration such asrust or peeling of coating which appears on the surface of a structureincreases the possibility of causing, in the near future, a failure. Theidentification of such an abnormal portion on the surface oftencurrently relies on visual determination, and therefore, the importanceof a system for automatically identifying the abnormal portion isgrowing from the viewpoint of oversight, determination based onsubjectivity, and additional processes of dispatching inspectors. Alaser distance measurement device is a device capable of acquiring athree-dimensional structure of an object in the surroundings of thedevice, and often has a function of measuring received light brightnessof laser light in addition to the information of a three-dimensionalobject point. In general, the received light brightness, i.e.,reflection brightness from the object depends on the state of the objectsurface to which the laser light is emitted. This enables a portion ofan abnormality such as rust or peeling of coating on the surface to bedetected through the processing of the received light brightnessinformation acquired by the laser distance measurement device.Hereinafter, in the present disclosure, the received light brightnessacquired by the laser distance measurement device is referred to as“reflection brightness.”

Patent Literature 1 discloses that a minimum curvature directionestimation unit estimates a minimum curvature direction for each region,an autocorrelation value calculation unit calculates an autocorrelationvalue of a feature amount of a partial region for each region, a sweepshape candidate region determination unit determines that each region inwhich the autocorrelation value is larger than a threshold is a sweepshape candidate region, a region integration processing unit integratesthe regions determined to be the sweep shape candidate regions, and asweep shape determination unit determines whether the integrated regionhas a sweep shape.

In Non Patent Literature 1, the roughness of a surface to be observed ismeasured based on position information of a point cloud. To obtain thesurface roughness, an average curved surface is locally calculated, sothat the displacement of the point cloud from the plane can becalculated as roughness. In Non Patent Literature 1, a roughness valueof the ground surface observed from the distance measurement device ofan aircraft is measured with centimeter (cm) scale accuracy.

In Non Patent Literature 2 and Non Patent Literature 3, the recognitionof an object under measurement and the determination of materials areattempted by focusing on the information of reflection brightness of thepoint cloud. There is adopted a method of correcting the reflectionbrightness acquired by the laser distance measurement device by assumingmodeling based on the radar equation and modeling of the bidirectionalreflectance distribution function for the reflection brightness of theacquired point cloud.

CITATION LIST Patent Literature

Patent Literature 1

-   Japanese Unexamined Patent Application Publication No. 2016-118502    Non Patent Literature 1-   R. Turner, et. al, “Estimation of soil surface roughness of    agricultural soils using airborne LiDAR”, Remote Sens. Envrion.    2014, 140, 107-117, (2014) Non Patent Literature 2-   S. Kaasalainen, et. al, “Radiometric Calibration of LIDAR Intensity    With Commercially Available Reference Targets”, IEEE Transactions on    Geoscience and Remote Sensing, vol. 47, pp. 588-598, (2009) Non    Patent Literature 3-   X. Li and Y. Liang, “Remote measurement of surface roughness,    surface reflectance, and body reflectance with LiDAR”, Appl. Opt.    54(30), 8904-8912, (2015)

SUMMARY OF INVENTION Technical Problem

Patent Literature 1 does not disclose that the abnormal portion on thesurface of the structure is detected. In the method in Non PatentLiterature 1, it is impossible to detect the roughness below thedistance measurement accuracy of the laser distance measurement device.The deterioration such as rust or peeling of coating on the surface is adisplacement that is below approximately 1 millimeter (mm), and is finerthan the accuracy of the laser distance measurement device which iswidely used at present. Therefore, it is difficult to identify theseabnormal portions on the surface from the roughness based on theposition information of the point cloud. In the above-described methodsin Non Patent Literature 2 and Non Patent Literature 3, it is difficultto identify the abnormality on the surface of the equipment in thefacility in which the equipment having a complex structure is disorderlyarranged. In such a facility, since the absorption property andreflection anisotropy of the laser light are different from surface tosurface of the equipment, it is difficult to determine the abnormalportion based on the information of only the reflection brightness. Forexample, when the abnormal portion is determined based on a uniformthreshold for the reflection brightness, all of the used surfacematerials having a strong absorption of the laser light wavelength aredetermined as the abnormal portions. In addition, the modeling of thereflection and absorption properties with respect to the all thesurfaces of the equipment in the facility is not a realistic method.

To solve any one of the above-described problems, an object of thepresent disclosure is to provide a surface abnormality detection device,a system, and a method.

Solution to Problem

A surface abnormality detection device according to the presentdisclosure includes:

a classification means for classifying an object under measurement intoone or more clusters having the same structure, based on positioninformation at a plurality of points on a surface of the object undermeasurement;

a determination means for determining a reflection brightness normalvalue of the cluster based on a distribution of reflection brightnessvalues at a plurality of points on a surface of the cluster; and

an identification means for identifying an abnormal portion on thesurface of the cluster based on a difference between the reflectionbrightness normal value and the reflection brightness value at each ofthe plurality of points on the surface of the cluster.

A surface abnormality detection device according to the presentdisclosure includes:

a first calculation means for calculating a first incident angle of alaser for each of a plurality of distance measurement points based onposition information of a first observation point, and positioninformation, included in first point cloud data, of the plurality ofdistance measurement points of a surface of an object under measurement;

a second calculation means for calculating a second incident angle of alaser for each of the plurality of distance measurement points based onposition information of a second observation point, and positioninformation of the plurality of distance measurement points included insecond point cloud data;

a position control means for making an adjustment to match positions foreach of the plurality of distance measurement points based on theposition information of the plurality of distance measurement points inthe first point cloud data and the position information of the pluralityof distance measurement points in the second point cloud data;

a brightness difference calculation means for calculating, for each ofthe plurality of distance measurement points, a reflection brightnessdifference value which is a difference between a first reflectionbrightness value at each of the plurality of distance measurement pointsin the first point cloud data after the position adjustment and a secondreflection brightness value at each of the plurality of distancemeasurement points in the second point cloud data after the positionadjustment;

a correction means for calculating, for each of the plurality ofdistance measurement points, an incident angle difference which is adifference between the first incident angle at each of the plurality ofdistance measurement points in the first point cloud data after theposition adjustment and the second incident angle at each of theplurality of distance measurement points in the second point cloud dataafter the position adjustment, and correcting, for each of the pluralityof distance measurement points, the reflection brightness differencevalue based on the incident angle difference; and

an identification means for identifying an abnormal portion of theobject under measurement based on the reflection brightness differencevalue after the correction.

A surface abnormality detection device according to the presentdisclosure includes:

a position control means for making an adjustment to match positions foreach of a plurality of distance measurement points based on positioninformation of the plurality of distance measurement points on a surfaceof an object under measurement, the position information being includedin cloud data for evaluation and position information of the pluralityof distance measurement points included in cloud data for comparison;

a brightness difference calculation means for calculating, for each ofthe plurality of distance measurement points, a reflection brightnessdifference value which is a difference between a reflection brightnessvalue for evaluation at each of the plurality of distance measurementpoints in the cloud data for evaluation after the position adjustmentand a reflection brightness value for comparison at each of theplurality of distance measurement points in the cloud data forcomparison after the position adjustment; and

an identification means for identifying an abnormal portion of theobject under measurement based on the reflection brightness differencevalue.

A surface abnormality detection device according to the presentdisclosure includes:

a classification means for evaluation for classifying an object undermeasurement into one or more clusters having the same structure, basedon position information at a plurality of distance measurement points ona surface of the object under measurement included in cloud data forevaluation;

a classification means for comparison for classifying the object undermeasurement into one or more clusters having the same structure, basedon position information at the plurality of distance measurement pointsincluded in cloud data for comparison;

a determination means for comparison for determining a reflectionbrightness normal value for each cluster of the cloud data forcomparison based on a distribution of reflection brightness values atthe plurality of distance measurement points of the cluster of the clouddata for comparison;

a control means for associating the cluster of the cloud data forevaluation with the cluster of the cloud data for comparison recognizedas having the same structure, based on the position information of theplurality of distance measurement points of the cluster of the clouddata for evaluation and the position information of the plurality ofdistance measurement points of the cluster of the cloud data forcomparison;

a calculation means for calculating a reflection brightness normaldifference value which is a difference between the reflection brightnessvalue at each of the plurality of distance measurement points of thecluster of the cloud data for evaluation and the reflection brightnessnormal value of the cluster of the cloud data for comparisoncorresponding to the cluster of the cloud data for evaluation; and

an identification means for identifying, for each cluster, an abnormalportion on the surface of the object under measurement based on thereflection brightness normal difference value.

A system according to the present disclosure includes:

a measurement device; and a surface abnormality detection device,

wherein

the measurement device acquires

a reflection brightness value at each of a plurality of points on asurface of an object under measurement, and

the surface abnormality detection device includes:

-   -   a classification means for classifying the object under        measurement into one or more clusters having the same structure,        based on position information at a plurality of points on the        surface of the object under measurement;    -   a determination means for determining a reflection brightness        normal value of the cluster based on a distribution of        reflection brightness values at a plurality of points on a        surface of the cluster; and    -   an identification means for identifying an abnormal portion on        the surface of the cluster based on a difference between the        reflection brightness normal value and the reflection brightness        value at each of the plurality of points on the surface of the        cluster.

Advantageous Effects of Invention

According to the present disclosure, there can be provided a surfaceabnormality detection device, and a system, capable of detecting anabnormal portion having a displacement below the distance measurementaccuracy when detecting the abnormal portion on the surface of astructure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a surface abnormality detectiondevice according to a first example embodiment.

FIG. 2 is a block diagram illustrating a system according to the firstexample embodiment.

FIG. 3 is a diagram illustrating the classification of an object undermeasurement into the same structure, according to the first exampleembodiment.

FIG. 4 is a diagram illustrating an abnormal portion in the classifiedstructure, according to the first example embodiment.

FIG. 5 is a flowchart illustrating an operation of the surfaceabnormality detection device according to the first example embodiment.

FIG. 6 is a flowchart illustrating an operation of a surface abnormalitydetection device according to a second example embodiment.

FIG. 7 is a diagram illustrating a laser incident angle according to athird example embodiment.

FIG. 8 is a flowchart illustrating an operation of a surface abnormalitydetection device according to the third example embodiment.

FIG. 9 is a diagram illustrating further classification (division) of acluster based on a laser incident angle according to a fourth exampleembodiment.

FIG. 10 is a flowchart illustrating an operation of a surfaceabnormality detection device according to the fourth example embodiment.

FIG. 11 is a flowchart illustrating an operation of a surfaceabnormality detection device according to a fifth example embodiment.

FIG. 12 is a flowchart illustrating an operation of a surfaceabnormality detection device according to a sixth example embodiment.

FIG. 13 is a diagram illustrating a distance measurement device and anobject under measurement according to a seventh example embodiment.

FIG. 14 is a block diagram illustrating a surface abnormality detectiondevice according to the seventh example embodiment.

FIG. 15 is a flowchart illustrating an operation of the surfaceabnormality detection device according to the seventh exampleembodiment.

FIG. 16 is a block diagram illustrating a surface abnormality detectiondevice according to an eighth example embodiment.

FIG. 17 is a flowchart illustrating an operation of the surfaceabnormality detection device according to the eighth example embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, example embodiments of the present invention will bedescribed with reference to the drawings. Throughout the drawings, thesame or corresponding components are denoted by the same referencesymbols and overlapping description will be omitted as appropriate forthe sake of clarity of the description.

First Example Embodiment

An overview of a surface abnormality detection device and a systemaccording to a first example embodiment will be described.

FIG. 1 is a block diagram illustrating the surface abnormality detectiondevice according to the first example embodiment.

FIG. 2 is a block diagram illustrating the system according to the firstexample embodiment.

As illustrated in FIG. 1, a surface abnormality detection device 11according to the first example embodiment includes a classificationmeans 111, a determination means 112, and an identification means 113.

The classification means 111 classifies an object under measurement intoa cluster having the same structure, based on position information at aplurality of points on a surface of the object under measurement. Theposition information can be represented as position information onthree-dimensional coordinates, for example.

The determination means 112 determines a reflection brightness normalvalue of the cluster based on the distribution of reflection brightnessvalues at the plurality of points on the surface of the cluster. In thecase where there are a plurality of classified clusters, thedetermination means 112 determines a reflection brightness normal valuefor each of the plurality of clusters.

The identification means 113 identifies an abnormal portion on thesurface of the cluster based on a difference between the reflectionbrightness normal value and the reflection brightness value at each ofthe plurality of points on the surface of the cluster. In the case wherethere are a plurality of classified clusters, the identification means113 identifies an abnormal portion on the surface for each of theplurality of clusters.

The data including the position information at the plurality of pointson the surface of the object under measurement and the reflectionbrightness value at each position is referred to as point cloud data ofthe object under measurement. The data including the positioninformation at the plurality of points on the surface of the cluster andthe reflection brightness value at each position is referred to as pointcloud data of the cluster. However, the cluster will be described later.

As illustrated in FIG. 2, a system 10 according to the first exampleembodiment includes a distance measurement device 12 and the surfaceabnormality detection device 11. The distance measurement device may bealso referred to as a measurement device.

The distance measurement device 12 includes a laser distance measurementdevice or the like, and acquires three-dimensional shape data of asurrounding object including the object under measurement. The surfaceabnormality detection device 11 acquires the three-dimensional shapedata from the distance measurement device 12 and identifies a portionwhere the surface state is abnormal, from the acquired three-dimensionalshape data.

In the following description of the example embodiment, thethree-dimensional shape data acquired by the distance measurement device12 (laser distance measurement device) is acquired as “three-dimensionalpoint cloud data” including the position information onthree-dimensional coordinates of the plurality of points on the surfaceof the object (object under measurement) and the information of thereflection brightness (reflection brightness value) at each position. Inthe description of the example embodiment, the processing on the“three-dimensional point cloud data” will be described, but the presentinvention is not limited thereto. The example embodiment is applicableto the point cloud data which includes the space information capable ofidentifying three-dimensional coordinates and the reflection brightnessat the coordinates (position). The three-dimensional point cloud datamay be also referred to as three-dimensional data or point cloud data.In addition, the plurality of points on the surface of the object undermeasurement may be also referred to as a point cloud.

Here, an overview of the classification of an object under measurementinto the same structure and the determination of an abnormal portionwill be described.

FIG. 3 is a diagram illustrating the classification of the object undermeasurement into the same structure, according to the first exampleembodiment.

FIG. 4 is a diagram illustrating an abnormal portion in the classifiedstructure, according to the first example embodiment.

A point cloud PC10 illustrated in FIG. 3 indicates a point cloud on thesurface of the object under measurement. The distance measurement device12 acquires the point cloud data including the position information onthree-dimensional coordinates and the reflection brightness in the pointcloud PC10. That is, the point cloud data acquired by the distancemeasurement device 12 is expressed as in the point cloud PC10 as a setof points including the position information on three-dimensionalcoordinates and the reflection brightness. Since a plurality ofstructures whose surface states are different in paint or the like existin the point cloud PC10, it is difficult to determine the abnormalportion on the surface by a uniform reflection brightness value.

Therefore, the surface abnormality detection device 11 according to thefirst example embodiment divides and classifies the point cloudconstituting the same structure into clusters by a clustering processbased on the position information on three-dimensional coordinates. Apoint cloud PC11 illustrated in FIG. 3 indicates a part of clustersafter the clustering process. A cluster C101 and a cluster C102 of thepoint cloud PC11 are point clouds which are determined and classified asdifferent structures. However, examples of algorithms of the clusteringinclude a method of determining the same cluster by using Euclideandistance as a threshold, and a region growth method of determining thesame cluster based on the continuity of angles of perpendicular linesamong neighboring points.

A point cloud PC12 illustrated in FIG. 4 indicates a partial clusterC102 p, which is a cluster which includes many point clouds in which thereflection brightness value is below a predetermined threshold, in thecluster C102. The partial cluster C102 p includes many point clouds inwhich the reflection brightness is weaker than the others. Note that thepoint clouds other than the cluster C102 are not illustrated forsimplification purposes.

A reflection brightness distribution G11 illustrated in FIG. 4represents the reflection brightness values of the point cloudcorresponding to the cluster C102 by a histogram. In the cluster C102,for example, a portion (abnormal portion) whose surface has become roughdue to rust has the reflection brightness that is lower than that at theother portions.

As an example in which such an abnormal portion is determined, there isa method of calculating an approximate curve L101 with respect to thehistogram of the portion in which the paint remains, and determining thepoint cloud having the reflection brightness deviating from theapproximate curve L101 as the point cloud having the abnormal surface.This determination method enables separation of a histogram H101 havingnormal reflection brightness values from a histogram H102 havingabnormal reflection brightness. The abnormal portion in the structurecan be determined by recognizing the point cloud corresponding to thehistogram H102 as the abnormal portion.

An operation of the surface abnormality detection device according tothe first example embodiment will be described.

FIG. 5 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the first example embodiment.

As illustrated in FIG. 5, the surface abnormality detection device 11acquires the three-dimensional point cloud data (step S101).

The surface abnormality detection device 11 performs the clusteringprocess based on the position information of the three-dimensional pointcloud data, and classifies the three-dimensional point cloud data intothe point cloud having the same structure, i.e., the cluster (stepS102).

The surface abnormality detection device 11 determines a normal value ofthe reflection brightness of the point cloud (cluster) classified intothe same structure, based on the reflection brightness distribution ofthe point cloud (step S103). In the case where there are a plurality ofpoint clouds classified into the same structure, a normal value of thereflection brightness is determined for each of the plurality of pointclouds. The normal value of the reflection brightness is referred to asa reflection brightness normal value.

When the deviation of the reflection brightness value of a point cloudfrom the reflection brightness normal value determined in step S103exceeds the threshold (step S104: YES), the surface abnormalitydetection device 11 determines the point cloud as the abnormal portionon the surface (step S105). That is, when, a difference between thereflection brightness value of a point cloud among a plurality of pointson the surface of a point cloud (cluster) and the reflection brightnessnormal value of the point cloud exceeds the threshold, the surfaceabnormality detection device 11 determines, as the abnormal portion, thepoint cloud (or the point) in this case.

When the deviation of the reflection brightness value of the point cloudfrom the reflection brightness normal value determined in step S103 isbelow the threshold (step S104: NO), the surface abnormality detectiondevice 11 determines the point cloud as the normal portion on thesurface (step S106). That is, when, a difference between the reflectionbrightness value of a point cloud among a plurality of points on thesurface of a point cloud (cluster) and the reflection brightness normalvalue of the point cloud is below the threshold, the surface abnormalitydetection device 11 determines, as the normal portion, the point cloud(or the point) in this case.

Thus, the surface abnormality detection device 11 of the first exampleembodiment can identify the abnormal portion on the surface from thethree-dimensional point cloud data including the reflection brightness.This makes it possible to identify the abnormal portion for the surfaceroughness finer than the distance measurement accuracy of the distancemeasurement device 12, and reduce false detection. Furthermore, aportion where the reflection brightness is abnormal is identified on aper structure basis, whereby the abnormal portion can be identified forthe structures whose surface states are different.

As a result, there can be provided a surface abnormality detectiondevice, and a system, capable of detecting an abnormal portion having adisplacement below the distance measurement accuracy when detecting theabnormal portion on the surface of a structure.

Second Example Embodiment

A surface abnormality detection device 21 according to a second exampleembodiment is different from the surface abnormality detection device 11according to the first example embodiment in that the reflectionbrightness attenuation caused according to the distance between a pointcloud and an observation point is corrected for the reflectionbrightness of the point cloud.

When a part of the structure (object under measurement) extends in adepth direction as viewed from the observation point, a lightpropagation distance is different between near point cloud and far pointcloud on the surface of the structure. As a result, since theattenuation is caused by light absorption and light scattering, thereflection brightness changes. Therefore, as compared with the surfaceabnormality detection device 11, the surface abnormality detectiondevice 21 performs an additional process of correcting the reflectionbrightness according to the distance between the point cloud and theobservation point. In this way, the surface abnormality detection device21 can identify the abnormal portion with higher accuracy than in thesurface abnormality detection device 11.

An operation of the surface abnormality detection device 21 according tothe second example embodiment will be described.

FIG. 6 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the second example embodiment.

As illustrated in FIG. 6, the processes from step S101 to step S102 areperformed in a similar manner to those in the first example embodiment.After step S102, the surface abnormality detection device 21 correctsthe reflection brightness value of each point cloud based on thedistance from the observation point to the point cloud (step S201). Thatis, the reflection brightness value is corrected based on an attenuationamount due to the distance between the surface abnormality detectiondevice 21 which is the observation point and the point (point cloud) onthe surface of the cluster. The reflection brightness value is a valueobtained by correcting the attenuation amount based on the distancebetween the surface abnormality detection device 21 which is theobservation point and the point (point cloud) on the surface of thecluster. For example, the reflection brightness may be corrected byperforming the attenuation correction by the distance to the fourthpower according to the radar equation. In addition, for example, thereflection brightness may be corrected by using an estimation valuebased on absorption by a propagation medium.

After step S201, the processes from step S103 to step S106 are performedin a similar manner to those in the first example embodiment.

However, the example has been described in which step S201 is performedbetween step S102 and step S103, but is not limited thereto. Thesequence of processes may be arbitrary when the requirement that stepS201 is performed before step S103 is satisfied.

In this way, the surface abnormality detection device 21 according tothe second example embodiment can identify the abnormal portion on thesurface of the structure, in particular, the structure extending in thedepth direction, with higher accuracy than in the surface abnormalitydetection device 11 according to the first example embodiment.

Third Example Embodiment

FIG. 7 is a diagram illustrating a laser incident angle according to athird example embodiment.

A surface abnormality detection device 31 according to a third exampleembodiment is different from the surface abnormality detection device 11according to the first example embodiment in that a process ofcorrecting reflection brightness relative to the laser incident angle ateach point is added.

Regarding the reflected light of the laser from a surface of thestructure, the angular dependence of the reflection brightness changesaccording to the nature of the surface. The angular dependence of thereflection brightness of the laser reflected light changes according tothe nature of the surface of the structure. Therefore, when the surfaceof the structure is curved, the abnormal portion on the surface of thestructure can be identified with higher accuracy by correcting thereflection brightness.

A point cloud PC32 illustrated in FIG. 7 is a schematic view in which apoint cloud in a three-dimensional region R31 in a point cloud PC31 isenlarged. The description will be made by way of example where a pointcloud on a cylindrical pipe is used as the point cloud PC31.

As illustrated in FIG. 7, a laser incident angle A301 at a distancemeasurement point P301 is calculated as an angle formed by a laserincident direction B301 that connects the distance measurement pointP301 and an observation point (surface abnormality detection device 31)and a perpendicular line N301 at the distance measurement point P301.The perpendicular line N301 is calculated by using a distancemeasurement point cloud in the surroundings of the distance measurementpoint P301.

An operation of the surface abnormality detection device 31 according tothe third example embodiment will be described.

FIG. 8 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the third example embodiment.

As illustrated in FIG. 8, the processes from step S101 to step S102 areperformed in a similar manner to those in the first example embodiment.After step S102, the surface abnormality detection device 31 calculates(estimates) the laser incident angle A301 based on the laser incidentdirection at each point and the perpendicular line N301 at each point(step S301). When the perpendicular line N301 is calculated, theposition of the point cloud may be smoothed to reduce the dispersion ofthe perpendicular line N301 due to an error of the distance measurementpoint P301.

The surface abnormality detection device 11 corrects (attenuates) thereflection brightness value at each point (distance measurement point)based on the laser incident angle A301 (step S302). The reflectionbrightness value may be corrected by applying the known reflectanceproperty, other than modeling of the bidirectional reflectancedistribution function of the structure, or simple modeling assumingLambertian reflection.

After step S302, the processes from step S103 to step S106 are performedin a similar manner to those in the first example embodiment.

However, the example has been described in which step S301 and step S302are performed between step S102 and step S103, but is not limitedthereto. The sequence of processes may be arbitrary when the requirementthat step S301 and step S302 are performed before step S103 issatisfied.

In this way, the surface abnormality detection device 31 according tothe third example embodiment can identify the abnormal portion on thesurface of the structure, in particular, the curved structure, withhigher accuracy than in the surface abnormality detection device 11according to the first example embodiment.

Fourth Example Embodiment

FIG. 9 is a diagram illustrating further classification (division) of acluster based on a laser incident angle according to a fourth exampleembodiment.

A point cloud PC41 illustrated in FIG. 9 is a point cloud on acylindrical pipe.

A surface abnormality detection device 41 according to the fourthexample embodiment identifies an abnormal portion by further dividing apoint cloud in the cluster into point clouds having the same laserincident angle.

In the fourth example embodiment, the description will be made by way ofexample where a point cloud on a cylindrical pipe is used as the pointcloud PC41.

As illustrated in FIG. 9, the surface abnormality detection device 41according to the fourth example embodiment further classifies (divides)the cluster into subclusters according to the laser incident angle. Forexample, a subcluster SC401 into which the cluster is further classifiedincludes a point cloud with a wide laser incident angle, and asubcluster SC402 includes a point cloud with a wide laser incident anglenext to that of the subcluster SC401. In addition, a reflectionbrightness distribution G41 illustrated in FIG. 9 shows a histogram ofreflection brightness values in the subcluster SC401. A reflectionbrightness distribution G42 illustrated in FIG. 9 shows a histogram ofreflection brightness values in the subcluster SC402.

The surface abnormality detection device 41 according to the fourthexample embodiment extracts the abnormal value of the reflectionbrightness from each of the reflection brightness distribution G41 andthe reflection brightness distribution G42 in a similar manner to thesurface abnormality detection device 11 according to the first exampleembodiment. That is, the surface abnormality detection device 41extracts, from each of the reflection brightness distribution G41 andthe reflection brightness distribution G42, the point cloud determinedas the abnormal portion in which a difference between the reflectionbrightness value of the point cloud and the reflection brightness normalvalue exceeds the threshold. This makes it possible to identify ahistogram H422 in which the reflection brightness becomes the abnormalvalue.

Specifically, with respect to the reflection brightness distribution G41and the reflection brightness distribution G42, the reflectionbrightness distributions having the normal value are calculated as anapproximate curve L411 and an approximate curve L421, respectively. Ahistogram H411 and a histogram H421 in which the reflection brightnessvalue becomes the normal value are identified by calculating theapproximate curve L411 and the approximate curve L421.

An operation of the surface abnormality detection device 41 according tothe fourth example embodiment will be described.

FIG. 10 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the fourth example embodiment.

As illustrated in FIG. 10, the processes from step S101 to step S102 areperformed in a similar manner to those in the first example embodiment.After step S102, step S301 is performed in a similar manner to that inthe third example embodiment. After step S301, the surface abnormalitydetection device 41 further classifies the point cloud (cluster)classified as the same structure into the subclusters according to avalue of the laser incident angle (step S404). For example, the surfaceabnormality detection device 41 further classifies the cluster into thesubcluster for each angle range of the laser incident angles in thepoint cloud of the cluster. When there are a plurality of point clouds,each point cloud is further classified according to the value of thelaser incident angle.

The point cloud may be further classified by a fixed width with respectto the value of the laser incident angle. Alternatively, the point cloudmay be further classified by a width varying according to the reflectionmodel or the number of points of the point cloud, with respect to thevalue of the laser incident angle.

The surface abnormality detection device 11 determines the normal valueof the reflection brightness in each of the point clouds (subclusters)classified as being included in the same cluster (the same structure)and as having the same laser incident angle, based on the reflectionbrightness distribution of the point cloud (step S402).

After step S402, the processes from step S103 to step S106 are performedin a similar manner to those in the first example embodiment.

The surface abnormality detection device 11 finally identifies theabnormal portion on the surface of the further classified point cloud(subcluster) based on the difference between the reflection brightnessnormal value of the classified point cloud (subcluster) and thereflection brightness value at each of the plurality of points on thesurface of the classified point cloud (subcluster).

However, the example has been described in which step S301 is performedbetween step S102 and step S401, but is not limited thereto. Thesequence of processes may be arbitrary when the requirement that stepS301 is performed before step S404 is satisfied.

In this way, the surface abnormality detection device 41 according tothe fourth example embodiment can identify the abnormal portion on thesurface with higher accuracy from the three-dimensional point cloud datahaving the reflection brightness in particular in a case where it isdifficult to correct the reflection brightness using the laser incidentangle with respect to the curved structure.

The surface abnormality detection device 41 according to the fourthexample embodiment can identify the abnormal portion on the surface withhigher accuracy than in the surface abnormality detection device 11according to the third example embodiment.

Fifth Example Embodiment

A surface abnormality detection device 51 according to a fifth exampleembodiment can determine an abnormal portion on a surface with higheraccuracy by using identification of the abnormal portion on the surfacethat is determined based on a red-green-blue (RGB) value in addition tothe identification of the abnormal portion on the surface based on thereflection brightness.

An operation of the surface abnormality detection device 51 according tothe fifth example embodiment will be described.

FIG. 11 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the fifth example embodiment.

As illustrated in FIG. 11, the surface abnormality detection device 51acquires three-dimensional point cloud data including RGB information(step S501).

The surface abnormality detection device 51 performs the clusteringprocess based on the position information of the three-dimensional pointcloud data, and classifies the three-dimensional point cloud data intothe point cloud having the same structure, i.e., the cluster (stepS502).

The surface abnormality detection device 51 determines the abnormalportion where the reflection brightness value becomes the abnormal valuein the point cloud (cluster) classified as having the same structure,based on the reflection brightness distribution of the point cloud (stepS503). When there are a plurality of point clouds, a portion where thereflection brightness value is abnormal is determined for each of theplurality of point clouds.

The surface abnormality detection device 51 determines a portion wherethe RGB value is abnormal in the point cloud classified as having thesame structure, based on the RGB value of the point cloud (step S504).When there are a plurality of point clouds, a portion where the RGBvalue is abnormal is determined for each of the plurality of pointclouds. The portion where the RGB value is abnormal may be determined ina similar procedure to step S503 after conversion to grayscale.

That is, the surface abnormality detection device 51 determines an RGBnormal value of the cluster based on the distribution of the RGB valuesat the plurality of points on the surface of the point cloud (cluster).Then, the surface abnormality detection device 51 identifies theabnormal portion on the surface of the cluster based on the differencebetween the RGB normal value and the RGB value at each of the pluralityof points on the surface of the cluster.

The surface abnormality detection device 51 identifies a desiredabnormal portion based on the abnormal portion determined based on thereflection brightness and the abnormal portion determined based on theRGB value (step S505).

In step S505, the abnormal portion determined based on the reflectionbrightness and the abnormal portion determined based on the RGB valuemay be complementarily used.

Examples of a difference between the detection using the reflectionbrightness value and the detection using the RGB value include a rustfluid. The rust fluid is determined as the abnormal portion based on theRGB value, but is not determined as the abnormal portion based on thereflection brightness value. Therefore, the outflow source can beidentified. The information can be used to identify the portion wherethe outflow source which is an original deterioration portion readilyoccurs, and an outflow path of the rust fluid, thereby enablingselection and determination of the appropriate repair method accordingto the degree of abnormality.

In this way, the surface abnormality detection device 51 according tothe fifth example embodiment can determine the abnormal portion on thesurface with higher accuracy from the three-dimensional point cloud datahaving the reflection brightness value and the RGB value.

Sixth Example Embodiment

A surface abnormality detection device 61 according to a sixth exampleembodiment can further improve the accuracy with which an abnormalportion on a surface is determined (identified), using theidentification of an abnormal portion on the surface which is determinedbased on the roughness, in addition to the identification of an abnormalportion on the surface based on a reflection brightness value.

For example, the spatial surface roughness can be calculated as adisplacement of the point cloud from the smoothed surface. Theabnormality on the surface that is rougher than the accuracy of thedistance measurement device 12 can be identified by identifying theabnormal portion on the surface based on the roughness. The abnormalportion on the surface can be complementarily identified by using theportion where the reflection brightness is abnormal and the portionwhere the roughness is abnormal.

An operation of the surface abnormality detection device 61 according tothe sixth example embodiment will be described.

FIG. 12 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the sixth example embodiment.

As illustrated in FIG. 12, the surface abnormality detection device 61acquires three-dimensional point cloud data (step S601).

The surface abnormality detection device 61 performs the clusteringprocess based on the position information of the three-dimensional pointcloud data, and classifies the three-dimensional point cloud data intothe point cloud having the same structure, i.e., the cluster (stepS602).

The surface abnormality detection device 61 calculates a roughness valueat each point based on the surrounding point cloud (step S603). That is,the surface abnormality detection device 61 calculates the roughnessvalue at each of the plurality of points on the surface of the clusterbased on the position information at the plurality of points on thesurface of the cluster. For example, the smoothed surface is calculatedbased on the point cloud in the surroundings of an arbitrary point P,and the displacement of the point P from the smoothed surface iscalculated as the roughness value of the point P.

The surface abnormality detection device 61 determines the abnormalportion where the reflection brightness value becomes the abnormal valuein the point cloud (cluster) classified as having the same structure,based on the reflection brightness distribution of the point cloud (stepS604).

The surface abnormality detection device 61 determines the portion wherethe roughness value is abnormal in the point cloud classified as havingthe same structure, based on the roughness value of the point cloud(step S605). When there are a plurality of point clouds, a portion wherethe roughness value is abnormal is determined for each of the pluralityof point clouds.

That is, the surface abnormality detection device 61 determines aroughness normal value of the cluster based on the distribution of theroughness values at the plurality of points on the surface of thecluster. Then, the surface abnormality detection device 61 identifiesthe abnormal portion on the surface of the cluster based on thedifference between the roughness normal value and the roughness value ateach of the plurality of points on the surface of the cluster.

The surface abnormality detection device 61 identifies a desiredabnormal portion based on the abnormal portion determined based on thereflection brightness value and the abnormal portion determined based onthe roughness value (step S606).

In step S606, the abnormal portion determined based on the reflectionbrightness and the abnormal portion determined based on the roughnessvalue may be complementarily used.

Examples of a difference between the detection using the reflectionbrightness value and the detection using the roughness value includelifting of coating due to internal corrosion. The lifting of coatingcauses no change to the reflection brightness value since paint remains,but is detected as the roughness value. The information can be used toidentify the penetration range of corrosion from the internal corrosionportion connected to the rust exposed to the outside, thereby enablingselection and determination of the appropriate repair method.

In this way, the surface abnormality detection device 61 according tothe sixth example embodiment can generally determine the abnormalportion on the surface, even with respect to the target (structure) thatis rougher than the accuracy of the distance measurement device 12.

Seventh Example Embodiment

FIG. 13 is a diagram illustrating a distance measurement device and anobject under measurement according to a seventh example embodiment.

Here, an operation of identifying an abnormal portion on a surface usingthe point cloud data obtained by capturing images from a plurality ofviewpoints will be described with reference to FIG. 13.

FIG. 13 illustrates an object under measurement T71 and an object undermeasurement T72, the images of which are captured with a distancemeasurement device 12 installed at a first observation point S71. Inaddition, FIG. 13 illustrates the object under measurement T71 and theobject under measurement T72, the images of which are captured with adistance measurement device 12 installed at a second observation pointS72.

As illustrated in FIG. 13, the data (information) at a distancemeasurement point P71 n on the surface of the object under measurementT71 is acquired by the distance measurement device 12 installed at eachof the first observation point S71 and the second observation point S72.In addition, the data (information) at a distance measurement point P72n on the surface of the object under measurement T72 is acquired by thedistance measurement device 12 installed at each of the firstobservation point S71 and the second observation point S72.

At this time, with respect to the distance measurement point P71 n, themeasurement is made at a laser incident angle A711 from the firstobservation point S71, and the measurement is made at a laser incidentangle A712 from the second observation point S72. Similarly, withrespect to the distance measurement point P72 n, the measurement is madeat a laser incident angle A721 from the first observation point S71, andthe measurement is made at a laser incident angle A722 from the secondobservation point S72.

In the case where an image of the object under measurement T71 iscaptured from the first observation point S71 and the second observationpoint S72, the laser incident angle on the distance measurement pointP71 n from the first observation point S71 is different from that fromthe second observation point S72, and therefore the reflectionbrightness value varies depending on the observation point. The same istrue for the case where an image of the measure target T72 is capturedfrom the first observation point S71 and the second observation pointS72.

In the seventh example embodiment, the abnormal portion on the surfaceis identified by focusing on the fact that the isotropic nature of thereflected light varies depending on the surface roughness.

In the case where the surface of the object under measurement is roughdue to the deterioration such as rust, the laser reflected light tendsto spread isotropically, and therefore a laser incident angle-dependentchange in the reflection brightness value is small. On the other hand,in the case where the surface of the object under measurement isprotected by the paint or the like, the laser reflected light has alarge reflection brightness value in a direction of specular reflection,and therefore a laser incident angle-dependent change in the reflectionbrightness value is large.

The abnormal portion on the surface of each of the measure target T71and the object under measurement T72 can be identified using adifference in reflection brightness acquired by the first observationpoint S71 and the second observation point S72, and a difference inlaser incident angle. Here, the difference in reflection brightness is adifference between the reflection brightness at a predetermined distancemeasurement point (e.g., the distance measurement point P71 n) acquiredby the first observation point S71 and the reflection brightness at thepredetermined distance measurement point acquired by the secondobservation point S72. In addition, the difference in laser incidentangle is a difference between the laser incident angle on thepredetermined distance measurement point from the first observationpoint S71 and the laser incident angle on the predetermined distancemeasurement point from the second observation point S72.

In the point cloud data of the images captured by the first observationpoint S71 and the second observation point S72, the positions of theobject under measurement T71 and the object under measurement T72 areassociated with the shapes thereof by position matching and recognition,respectively, and then the difference in reflection brightness and thedifference in laser incident angle are calculated, whereby the abnormalportion on the surfaces is identified.

However, the example has been described in which the reflectionbrightness at the same distance measurement point P71 n (or P72 n) isacquired from the first observation point S71 and the second observationpoint S72, but is not limited thereto. For example, in the case wherethe distance measurement point P71 n can be acquired by the firstobservation point S71, but the distance measurement point P71 n cannotbe acquired by the second observation point S72, a neighboring point ofthe distance measurement point P71 n or interpolation points of thereflection brightness and the laser incident angle at the distancemeasurement point P71 n may be used.

An overview of the surface abnormality detection device 71 according tothe seventh example embodiment will be described.

FIG. 14 is a block diagram illustrating the surface abnormalitydetection device according to the seventh example embodiment.

As illustrated in FIG. 14, the surface abnormality detection device 71according to the seventh example embodiment includes a first calculationmeans 714 a, a second calculation means 714 b, a position control means715, a brightness difference calculation means 716, a correction means717, and an identification means 713.

The first calculation means 714 a calculates a first incident angle ofthe laser for each of the plurality of distance measurement points basedon the position information of the first observation point S71 and theposition information of the plurality of distance measurement points onthe surface of the object under measurement. The position information ofthe first observation point S71 is included in first point cloud data.The first calculation means 714 a calculates the first incident angle atthe distance measurement point based on a direction connecting adistance measurement point on the surface of the object undermeasurement and the first observation point S71, and a perpendicularline at the distance measurement point.

The second calculation means 714 b calculates a second incident angle ofthe laser for each of the plurality of distance measurement points basedon the position information of the second observation point S72 and theposition information of the plurality of distance measurement pointsincluded in the second point cloud data. The second calculation means714 b calculates the second incident angle at a distance measurementpoint based on a direction connecting the distance measurement point onthe surface of the object under measurement and the second observationpoint S72, and a perpendicular line at the distance measurement point.

The position control means 715 adjusts to match the positions for eachof the plurality of distance measurement points based on the positioninformation of the plurality of distance measurement points in the firstpoint cloud data and the position information of the plurality ofdistance measurement points in the second point cloud data.

The brightness difference calculation means 716 calculates, for each ofthe plurality of distance measurement points, a reflection brightnessdifference value which is a difference between a first reflectionbrightness value at each of the plurality of distance measurement pointsin the first point cloud data after the position adjustment and a secondreflection brightness value at each of the plurality of distancemeasurement points in the second point cloud data after the positionadjustment.

The correction means 717 calculates, for each of the plurality ofdistance measurement points, an incident angle difference which is adifference between the first incident angle at each of the plurality ofdistance measurement points in the first point cloud data after theposition adjustment and the second incident angle at each of theplurality of distance measurement points in the second point cloud dataafter the position adjustment. The correction means 717 corrects, foreach of the plurality of distance measurement points, the reflectionbrightness difference value based on the calculated incident angledifference.

The identification means 713 identifies an abnormal portion of theobject under measurement based on the reflection brightness differencevalue after correction.

An operation of the surface abnormality detection device 71 according tothe seventh example embodiment will be described.

FIG. 15 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the seventh exampleembodiment.

In the following description of the example embodiment, thethree-dimensional point cloud data of the image captured by the firstobservation point S71 is referred to as the first point cloud data, andthe three-dimensional point cloud data of the image captured by thesecond observation point S72 is referred to as the second point clouddata.

As illustrated in FIG. 15, the surface abnormality detection device 71acquires the first point cloud data (step S701). The surface abnormalitydetection device 71 acquires the second point cloud data (step S702).

The surface abnormality detection device 71 calculates, with respect tothe first point cloud data, the first incident angle of the laser ateach distance measurement point based on the point cloud and theposition information of the first observation point S71 (step S703). Thesurface abnormality detection device 11 calculates, with respect to thesecond point cloud data, the second incident angle of the laser at eachdistance measurement point based on the point cloud and the positioninformation of the second observation point S72 (step S704). The laserincident angle is calculated as described in the third exampleembodiment, for example.

The surface abnormality detection device 71 performs the positionmatching between the first point cloud data and the second point clouddata, or the position matching between the first point cloud data andthe second point cloud data by shape identification (step S705).

The surface abnormality detection device 71 calculates a difference inreflection brightness value between the distance measurement pointscorresponding to each other (step S706). The difference in reflectionbrightness may be calculated using the points closest to each other inthe corresponding point cloud, or an interpolated value at thecorresponding position. In addition, the surface abnormality detectiondevice 71 also calculates a difference in laser incident angle betweenthe distance measurement points. The difference in reflection brightnessis referred to as a reflection brightness difference or a reflectionbrightness difference value.

The surface abnormality detection device 71 corrects the reflectionbrightness difference value based on the laser incident angle differencecalculated in step S706 (step S707). The laser incident angle differencemay be calculated using the points closest to each other in thecorresponding point cloud, or an interpolated value at the correspondingposition.

The reflection brightness difference value may be corrected by applyingthe known reflectance property, other than modeling of the bidirectionalreflectance distribution function of the object under measurement, orsimple modeling assuming Lambertian reflection.

When the reflection brightness difference value of a point cloud isbelow a difference threshold (step S708: YES), the surface abnormalitydetection device 71 determines the point cloud as an abnormal portion onthe surface (step S709).

When the reflection brightness difference value of a point cloud exceedsthe difference threshold (step S708: NO), the surface abnormalitydetection device 71 determines the point cloud as a normal portion onthe surface (step S710).

In this way, the surface abnormality detection device 71 according tothe seventh example embodiment can identify the abnormal portion on thesurface with higher accuracy from the three-dimensional point cloud dataof the images captured from a plurality of points.

Eighth Example Embodiment

A surface abnormality detection device 81 according to an eighth exampleembodiment is different from the surface abnormality detection device 71according to the seventh example embodiment in that an abnormal portionon the surface is identified by comparison to the three-dimensionalpoint cloud data measured in the past, whereby the accuracy is improved.

In the description of the eighth example embodiment, thethree-dimensional point cloud data for comparison measured in the pastis referred to as “three-dimensional point cloud data (comparison)” orcloud data for comparison, and the three-dimensional point cloud datafor evaluation for determining the abnormality is referred to as“three-dimensional point cloud data (evaluation)” or cloud data forevaluation.

The simplest method of comparing the three-dimensional point cloud data(comparison) with the three-dimensional point cloud data (evaluation) isa method of acquiring a reflection brightness difference value of thepoint cloud using the point of closest proximity between the pointclouds or the interpolation.

An overview of the surface abnormality detection device 81 according tothe eighth example embodiment will be described.

FIG. 16 is a block diagram illustrating the surface abnormalitydetection device according to the eighth example embodiment.

As illustrated in FIG. 16, the surface abnormality detection device 81according to the eighth example embodiment includes a position controlmeans 815, a brightness difference calculation means 816, and anidentification means 813.

In the cloud data for evaluation, the position information of aplurality of distance measurement points on the surface of the objectunder measurement is included. Also in the cloud data for comparison,the position information of a plurality of distance measurement pointson the surface of the object under measurement is included. The positioncontrol means 815 adjusts to match the positions for each of theplurality of distance measurement points based on the positioninformation of the plurality of distance measurement points included inthe cloud data for evaluation and the position information of theplurality of distance measurement points included in the cloud data forcomparison.

The brightness difference calculation means 816 calculates, for each ofthe plurality of distance measurement points, a reflection brightnessdifference value which is a difference between a reflection brightnessvalue for evaluation at each of the plurality of distance measurementpoints in the cloud data for evaluation after the position adjustmentand a reflection brightness value for comparison at each of theplurality of distance measurement points in the cloud data forcomparison after the position adjustment.

The identification means 813 identifies an abnormal portion of theobject under measurement based on the reflection brightness differencevalue.

In the following description of the eighth example embodiment, as amethod of identifying the abnormal portion on the surface with higheraccuracy, a method of determining the abnormal portion on a per clusterbasis, based on the position information of the point cloud will bedescribed as an example.

Specifically, a normal value of the reflection brightness value isdetermined on a per cluster basis from the three-dimensional point clouddata (comparison), the clusters are associated with each other betweenthe point clouds, and a deviation value of the reflection brightnessvalue is determined as a difference between a reflection brightnessvalue of the three-dimensional point cloud data (evaluation) and thereflection brightness normal value of the corresponding cluster, tothereby identify the abnormal portion. For example, in the case wherewhen two point clouds are measured, the distance measurement point ischanged due to an observation point error, or the like, the reflectionbrightness value may change. In such a case, the error can be reduced byprocessing the reflection brightness value on a per cluster basis.

An operation of the surface abnormality detection device 81 according tothe eighth example embodiment will be described.

FIG. 17 is a flowchart illustrating the operation of the surfaceabnormality detection device according to the eighth example embodiment.

As illustrated in FIG. 17, the surface abnormality detection device 81acquires the three-dimensional point cloud data (evaluation) (stepS801). The surface abnormality detection device 81 acquires thethree-dimensional point cloud data (comparison) (step S802).

The surface abnormality detection device 81 performs the clusteringprocess based on the position information of the point cloud withrespect to the three-dimensional point cloud data (evaluation), andclassifies the three-dimensional point cloud data (evaluation) into thepoint cloud having the same structure, i.e., the cluster (step S803).That is, the surface abnormality detection device 81 classifies theobject under measurement into one or more clusters having the samestructure, based on the position information at the plurality ofdistance measurement points on the surface of the object undermeasurement included in the three-dimensional point cloud data(evaluation).

The surface abnormality detection device 81 performs the clusteringprocess based on the position information of the point cloud withrespect to the three-dimensional point cloud data (comparison), andclassifies the three-dimensional point cloud data (comparison) into thepoint cloud having the same structure, i.e., the cluster (step S804).That is, the surface abnormality detection device 81 classifies theobject under measurement into one or more clusters having the samestructure, based on the position information at the plurality ofdistance measurement points on the surface of the object undermeasurement included in the three-dimensional point cloud data(comparison).

The surface abnormality detection device 81 calculates the normal valueof the reflection brightness (reflection brightness normal value) foreach classified cluster in step S804 (step S805). That is, the surfaceabnormality detection device 81 determines the reflection brightnessnormal value for each cluster of the three-dimensional point cloud data(comparison) based on the distribution of the reflection brightnessvalues in the plurality of distance measurement points of the cluster ofthe three-dimensional point cloud data (comparison).

The surface abnormality detection device 81 associates the clusterscorresponding to each other between the three-dimensional point clouddata (evaluation) and the three-dimensional potin cloud data(comparison) with the positions, respectively, by position matchingbetween the two pieces of three-dimensional point cloud data or shapeidentification of the clusters (step S806). That is, the surfaceabnormality detection device 81 associates the cluster of thethree-dimensional point cloud data (evaluation) with the cluster of thethree-dimensional point cloud data (comparison) recognized as having thesame structure, based on the position information of the plurality ofdistance measurement points of the cluster of the three-dimensionalpoint cloud data (evaluation) and the position information of theplurality of distance measurement points of the cluster of thethree-dimensional point cloud data (comparison).

The surface abnormality detection device 81 calculates a difference fromthe reflection brightness normal value based on the reflectionbrightness normal value calculated in step S805 (step S807). Thedifference from the reflection brightness normal value is referred to asa reflection brightness normal difference value. That is, the surfaceabnormality detection device 81 calculates the reflection brightnessnormal difference value which is a difference between the reflectionbrightness value at each of the plurality of distance measurement pointsof the cluster of the three-dimensional point cloud data (evaluation)and the reflection brightness normal value of the cluster of thethree-dimensional point cloud data (comparison) corresponding to thecluster of the three-dimensional point cloud data (evaluation).

When the reflection brightness normal difference value of a point cloudexceeds a normal difference threshold (step S808: YES), the surfaceabnormality detection device 81 determines the point cloud as anabnormal portion on the surface (step S809). That is, the surfaceabnormality detection device 81 identifies, for each cluster, theabnormal portion on the surface of the object under measurement based onthe reflection brightness normal difference value.

When the reflection brightness normal difference value of a point cloudis below the normal difference threshold (step S808: NO), the surfaceabnormality detection device 81 determines the point cloud as anabnormal portion on the surface (step S810).

In this way, the surface abnormality detection device 81 according tothe eighth example embodiment can identify the abnormal portion on thesurface with higher accuracy from the comparison with thethree-dimensional point cloud data of the image captured in the past.

As described above, according to the example embodiments, there can beprovided a processing device capable of reducing false detection when anabnormal matter is detected, a system, a method, and a non-transitorycomputer-readable medium.

Note that although the present invention has been described as ahardware configuration in the above-described example embodiments, thepresent invention is not limited the hardware configuration. In thepresent invention, the processes in each of the components can be alsoimplemented by causing a CPU (Central Processing Unit) to execute acomputer program.

In the above-described example embodiments, the program can be stored invarious types of non-transitory computer-readable media and therebysupplied to computers. The non-transitory computer-readable mediainclude various types of tangible storage media. Examples of thenon-transitory computer-readable media include a magnetic recordingmedium (such as a flexible disk, a magnetic tape, and a hard diskdrive), a magneto-optic recording medium (such as a magneto-optic disk),a CD-ROM (Read Only Memory), a CD-R, and a CD-R/W, and a semiconductormemory (such as a mask ROM, a PROM (Programmable ROM), an EPROM(Erasable PROM), a flash ROM, and a RAM (Random Access Memory)).Furthermore, the program can be supplied to computers by using varioustypes of transitory computer-readable media. Examples of the transitorycomputer-readable media include an electrical signal, an optical signal,and an electromagnetic wave. The transitory computer-readable media canbe used to supply programs to the computer through a wire communicationpath such as an electrical wire and an optical fiber, or wirelesscommunication path.

Although the present invention is explained above with reference toexample embodiments, the present invention is not limited to theabove-described example embodiments. Various modifications that can beunderstood by those skilled in the art can be made to the configurationand details of the present invention within the scope of the invention.

Note that the present invention is not limited to the above-describedexample embodiments and various changes may be made therein withoutdeparting from the spirit and scope of the present invention.

A part or the entire of the above-described example embodiments may bedescribed as the following supplementary notes, but not limited thereto.

(Supplementary Note 1)

A surface abnormality detection device, comprising:

a classification means for classifying an object under measurement intoone or more clusters having the same structure, based on positioninformation at a plurality of points on a surface of the object undermeasurement;

a determination means for determining a reflection brightness normalvalue of the cluster based on a distribution of reflection brightnessvalues at a plurality of points on a surface of the cluster; and

an identification means for identifying an abnormal portion on thesurface of the cluster based on a difference between the reflectionbrightness normal value and the reflection brightness value at each ofthe plurality of points on the surface of the cluster.

(Supplementary Note 2)

The surface abnormality detection device according to Supplementary Note1, wherein

the identification means determines, among the plurality of points onthe surface of the cluster, a predetermined point where the differencebetween the reflection brightness value and reflection brightness normalvalue exceeds a threshold value as the abnormal portion.

(Supplementary Note 3)

The surface abnormality detection device according to Supplementary Note1 or 2, wherein

the reflection brightness value is corrected based on an attenuationamount due to a distance between an own device which is an observationpoint and the point on the surface of the cluster.

(Supplementary Note 4)

The surface abnormality detection device according to any one ofSupplementary Note 1 to 3, wherein

a laser incident angle at a distance measurement point of the cluster iscalculated based on a direction connecting the distance measurementpoint of the cluster and the own device, and a perpendicular line at thedistance measurement point of the cluster, and

the reflection brightness value at the distance measurement point of thecluster is further corrected based on the laser incident angle.

(Supplementary Note 5)

The surface abnormality detection device according to Supplementary Note4, wherein

the classification means further classifies the cluster into subclustersbased on the laser incident angle,

the determination means determines a reflection brightness normal valueof the subcluster based on a distribution of reflection brightnessvalues at a plurality of points on a surface of the subcluster, and theidentification means identifies an abnormal portion on the surface ofthe subcluster based on a difference between the reflection brightnessnormal value of the subcluster and the reflection brightness value ateach of the plurality of points on the surface of the subcluster.

(Supplementary Note 6)

The surface abnormality detection device according to any one ofSupplementary Notes 1 to 5, wherein

the determination means determines an RGB normal value of the clusterbased on a distribution of RGB values at the plurality of points on thesurface of the cluster,

the identification means identifies an abnormal portion on the surfaceof the cluster based on a difference between the RGB normal value andthe RGB value at each of the plurality of points on the surface of thecluster, and

the identification means identifies a desired abnormal portion based onthe abnormal portion identified using the reflection brightness valueand the abnormal portion identified using the RGB value.

(Supplementary Note 7)

The surface abnormality detection device according to any one ofSupplementary Notes 1 to 5, wherein

a roughness value at each of the plurality of points on the surface ofthe cluster is calculated based on the position information at theplurality of points on the surface of the cluster,

the determination means determines a roughness normal value of thecluster based on a distribution of the roughness values at the pluralityof points on the surface of the cluster,

the identification means identifies an abnormal portion on the surfaceof the cluster based on a difference between the roughness normal valueand the roughness value at each of the plurality of points on thesurface of the cluster, and

the identification means identifies a desired abnormal portion based onthe abnormal portion identified using the reflection brightness valueand the abnormal portion identified using the roughness value.

(Supplementary Note 8)

A surface abnormality detection device, comprising:

a first calculation means for calculating a first incident angle of alaser for each of a plurality of distance measurement points based onposition information of a first observation point, and positioninformation, included in first point cloud data, of the plurality ofdistance measurement points of a surface of an object under measurement;

a second calculation means for calculating a second incident angle of alaser for each of the plurality of distance measurement points based onposition information of a second observation point, and positioninformation of the plurality of distance measurement points included insecond point cloud data;

a position control means for making an adjustment to match positions foreach of the plurality of distance measurement points based on theposition information of the plurality of distance measurement points inthe first point cloud data and the position information of the pluralityof distance measurement points in the second point cloud data;

a brightness difference calculation means for calculating, for each ofthe plurality of distance measurement points, a reflection brightnessdifference value which is a difference between a first reflectionbrightness value at each of the plurality of distance measurement pointsin the first point cloud data after the position adjustment and a secondreflection brightness value at each of the plurality of distancemeasurement points in the second point cloud data after the positionadjustment;

a correction means for calculating, for each of the plurality ofdistance measurement points, an incident angle difference which is adifference between the first incident angle at each of the plurality ofdistance measurement points in the first point cloud data after theposition adjustment and the second incident angle at each of theplurality of distance measurement points in the second point cloud dataafter the position adjustment, and correcting, for each of the pluralityof distance measurement points, the reflection brightness differencevalue based on the incident angle difference; and

an identification means for identifying an abnormal portion of theobject under measurement based on the reflection brightness differencevalue after the correction.

(Supplementary Note 9)

A surface abnormality detection device, comprising:

a position control means for making an adjustment to match positions foreach of a plurality of distance measurement points based on positioninformation of the plurality of distance measurement points on a surfaceof an object under measurement, the position information being includedin cloud data for evaluation and position information of the pluralityof distance measurement points included in cloud data for comparison;

a brightness difference calculation means for calculating, for each ofthe plurality of distance measurement points, a reflection brightnessdifference value which is a difference between a reflection brightnessvalue for evaluation at each of the plurality of distance measurementpoints in the cloud data for evaluation after the position adjustmentand a reflection brightness value for comparison at each of theplurality of distance measurement points in the cloud data forcomparison after the position adjustment; and

an identification means for identifying an abnormal portion of theobject under measurement based on the reflection brightness differencevalue.

(Supplementary Note 10)

A surface abnormality detection device, comprising:

a classification means for evaluation for classifying an object undermeasurement into one or more clusters having the same structure, basedon position information at a plurality of distance measurement points ona surface of the object under measurement included in cloud data forevaluation;

a classification means for comparison for classifying the object undermeasurement into one or more clusters having the same structure, basedon position information at the plurality of distance measurement pointsincluded in cloud data for comparison;

a determination means for comparison for determining a reflectionbrightness normal value for each cluster of the cloud data forcomparison based on a distribution of reflection brightness values atthe plurality of distance measurement points of the cluster of the clouddata for comparison;

a control means for associating the cluster of the cloud data forevaluation with the cluster of the cloud data for comparison recognizedas having the same structure, based on the position information of theplurality of distance measurement points of the cluster of the clouddata for evaluation and the position information of the plurality ofdistance measurement points of the cluster of the cloud data forcomparison;

a calculation means for calculating a reflection brightness normaldifference value which is a difference between the reflection brightnessvalue at each of the plurality of distance measurement points of thecluster of the cloud data for evaluation and the reflection brightnessnormal value of the cluster of the cloud data for comparisoncorresponding to the cluster of the cloud data for evaluation; and

an identification means for identifying, for each cluster, an abnormalportion on the surface of the object under measurement based on thereflection brightness normal difference value.

(Supplementary Note 11)

A system, comprising:

a measurement device configured to acquire a reflection brightness valueat each of a plurality of points on a surface of an object undermeasurement; and

the surface abnormality detection device according to any one ofSupplementary Notes 1 to 10,

wherein

the surface abnormality detection device

-   -   identifies an abnormal portion on the surface of the object        under measurement.

(Supplementary Note 12)

A method of a surface abnormality detection device, the methodcomprising:

classifying an object under measurement into one or more clusters havingthe same structure, based on position information at a plurality ofpoints on a surface of the object under measurement;

determining a reflection brightness normal value of the cluster based ona distribution of reflection brightness values at a plurality of pointson a surface of the cluster; and

identifying an abnormal portion on the surface of the cluster based on adifference between the reflection brightness normal value and thereflection brightness value at each of the plurality of points on thesurface of the cluster.

(Supplementary Note 13)

A non-transitory computer-readable medium storing a program configuredto cause a computer to execute:

classifying an object under measurement into one or more clusters havingthe same structure, based on position information at a plurality ofpoints on a surface of the object under measurement;

determining a reflection brightness normal value of the cluster based ona distribution of reflection brightness values at a plurality of pointson a surface of the cluster; and

identifying an abnormal portion on the surface of the cluster based on adifference between the reflection brightness normal value and thereflection brightness value at each of the plurality of points on thesurface of the cluster.

REFERENCE SIGNS LIST

-   10: SYSTEM-   11, 21, 31, 41, 51, 61, 71, 81: SURFACE ABNORMALITY DETECTION DEVICE-   111: CLASSIFICATION MEANS-   112: DETERMINATION MEANS-   113: IDENTIFICATION MEANS-   12: DISTANCE MEASUREMENT DEVICE-   713, 813: IDENTIFICATION MEANS-   714 a: FIRST CALCULATION MEANS-   714 b: SECOND CALCULATION MEANS-   715, 815: POSITION CONTROL MEANS-   716, 816: BRIGHTNESS DIFFERENCE CALCULATION MEANS-   717: CORRECTION MEANS-   PC10, PC11, PC12, PC31, PC32, PC41: POINT CLOUD-   R31: THREE-DIMENSIONAL REGION-   C101, C102: CLUSTER-   SC401, SC402: SUBCLUSTER-   G11, G41, G42: REFLECTION BRIGHTNESS DISTRIBUTION-   H101, H102, H411, H421, H422: HISTOGRAM-   L101, L411, L421: APPROXIMATE CURVE-   P301, P71 n, P72 n: DISTANCE MEASUREMENT POINT-   A301, A711, A712, A721, A722: LASER INCIDENT ANGLE-   N301: PERPENDICULAR LINE-   B301: LASER INCIDENT DIRECTION-   P: ARBITRARY POINT-   T71, T72: OBJECT UNDER MEASUREMENT-   S71: FIRST OBSERVATION POINT-   S72: SECOND OBSERVATION POINT

What is claimed is:
 1. A surface abnormality detection device,comprising: at least one memory storing instructions, and at least oneprocessor configured to execute the instructions to; classify an objectunder measurement into one or more clusters having the same structure,based on position information at a plurality of points on a surface ofthe object under measurement; determine a reflection brightness normalvalue of the cluster based on a distribution of reflection brightnessvalues at a plurality of points on a surface of the cluster; andidentify an abnormal portion on the surface of the cluster based on adifference between the reflection brightness normal value and thereflection brightness value at each of the plurality of points on thesurface of the cluster.
 2. The surface abnormality detection deviceaccording to claim 1, wherein the reflection brightness value iscorrected based on an attenuation amount due to a distance between anown device which is an observation point and the point on the surface ofthe cluster.
 3. The surface abnormality detection device according toclaim 1, wherein a laser incident angle at a distance measurement pointof the cluster is calculated based on a direction connecting thedistance measurement point of the cluster and the own device, and aperpendicular line at the distance measurement point of the cluster, andthe reflection brightness value at the distance measurement point of thecluster is further corrected based on the laser incident angle.
 4. Thesurface abnormality detection device according to claim 3, wherein theat least one processor further configured to execute the instructionsto; classify the cluster into subclusters based on the laser incidentangle, determine a reflection brightness normal value of the subclusterbased on a distribution of reflection brightness values at a pluralityof points on a surface of the subcluster, and identify an abnormalportion on the surface of the subcluster based on a difference betweenthe reflection brightness normal value of the subcluster and thereflection brightness value at each of the plurality of points on thesurface of the subcluster.
 5. The surface abnormality detection deviceaccording to claim 1, wherein the at least one processor furtherconfigured to execute the instructions to; determine an RGB normal valueof the cluster based on a distribution of RGB values at the plurality ofpoints on the surface of the cluster, identify an abnormal portion onthe surface of the cluster based on a difference between the RGB normalvalue and the RGB value at each of the plurality of points on thesurface of the cluster, and identify a desired abnormal portion based onthe abnormal portion identified using the reflection brightness valueand the abnormal portion identified using the RGB value.
 6. The surfaceabnormality detection device according to claim 1, wherein a roughnessvalue at each of the plurality of points on the surface of the clusteris calculated based on the position information at the plurality ofpoints on the surface of the cluster, the at least one processor furtherconfigured to execute the instructions to; determine a roughness normalvalue of the cluster based on a distribution of the roughness values atthe plurality of points on the surface of the cluster, identify anabnormal portion on the surface of the cluster based on a differencebetween the roughness normal value and the roughness value at each ofthe plurality of points on the surface of the cluster, and identify adesired abnormal portion based on the abnormal portion identified usingthe reflection brightness value and the abnormal portion identifiedusing the roughness value.
 7. A surface abnormality detection device,comprising: at least one memory storing instructions, and at least oneprocessor configured to execute the instructions to; calculate a firstincident angle of a laser for each of a plurality of distancemeasurement points based on position information of a first observationpoint, and position information, included in first point cloud data, ofthe plurality of distance measurement points of a surface of an objectunder measurement; calculate a second incident angle of a laser for eachof the plurality of distance measurement points based on positioninformation of a second observation point, and position information ofthe plurality of distance measurement points included in second pointcloud data; make an adjustment to match positions for each of theplurality of distance measurement points based on the positioninformation of the plurality of distance measurement points in the firstpoint cloud data and the position information of the plurality ofdistance measurement points in the second point cloud data; calculate,for each of the plurality of distance measurement points, a reflectionbrightness difference value which is a difference between a firstreflection brightness value at each of the plurality of distancemeasurement points in the first point cloud data after the positionadjustment and a second reflection brightness value at each of theplurality of distance measurement points in the second point cloud dataafter the position adjustment; calculate, for each of the plurality ofdistance measurement points, an incident angle difference which is adifference between the first incident angle at each of the plurality ofdistance measurement points in the first point cloud data after theposition adjustment and the second incident angle at each of theplurality of distance measurement points in the second point cloud dataafter the position adjustment, and correcting, for each of the pluralityof distance measurement points, the reflection brightness differencevalue based on the incident angle difference; and identify an abnormalportion of the object under measurement based on the reflectionbrightness difference value after the correction.
 8. A surfaceabnormality detection device, comprising: at least one memory storinginstructions, and at least one processor configured to execute theinstructions to; make an adjustment to match positions for each of aplurality of distance measurement points based on position informationof the plurality of distance measurement points on a surface of anobject under measurement, the position information being included incloud data for evaluation and position information of the plurality ofdistance measurement points included in cloud data for comparison;calculate, for each of the plurality of distance measurement points, areflection brightness difference value which is a difference between areflection brightness value for evaluation at each of the plurality ofdistance measurement points in the cloud data for evaluation after theposition adjustment and a reflection brightness value for comparison ateach of the plurality of distance measurement points in the cloud datafor comparison after the position adjustment; and identify an abnormalportion of the object under measurement based on the reflectionbrightness difference value.
 9. (canceled)
 10. A system, comprising: ameasurement device configured to acquire the reflection brightness valueat each of a plurality of points on a surface of an object undermeasurement; and the surface abnormality detection device according toclaim 1, wherein the surface abnormality detection device identifies anabnormal portion on the surface of the object under measurement.